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ABSTRACT 


This thesis was part of the ongoing researeh eondueted at the Naval Postgraduate 
Sehool to aehieve greater eollaboration between heterogeneous autonomous vehieles. The 
researeh addresses optimal eontrol issues in the eollaboration between an Unmanned 
Aerial Vehiele (UAV) and Autonomous Ground Vehieles (AGV). The seenario revolves 
around using the eamera onboard the UAV to extend the effeetive field of view of the 
AGV. For military operations, this eould be helpful in improving seeurity for eonvoys 
and riverine patrols. There were three main problems addressed in this thesis. The first 
problem dealt with the design of a UAV eontrol law that takes into eonsideration the 
relative speed differenees between the UAV and the AGV. The UAV was assumed to 
have a greater speed eompared to the AGV in this thesis. The seeond was the keystone 
field of view projeetion effeet of the UAV’s onboard eamera onto the earth. The image 
eaptured by the eamera was distorted due to the view angle of the eamera from a high 
elevation. The third problem addressed was eontrol of the loeation of the UAV to ensure 
the reliability of the eommunieation network between the UAV and the AGV. The 
eommunieation was assumed to be a linear funetion of the relative positions of the UAV 
and the AGV. 
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I. INTRODUCTION 


The rapid advancement of computer and sensor technology in recent years has 
resulted in an increased number of unmanned vehicles being employed in the battlefield 
to accomplish various dangerous tasks; one such application is the employment of 
unmanned vehicles for the detection and avoidance of mines and Improvised Explosive 
Devices (lED). The employment of unmanned vehicles for the above mentioned task 
usually has little interaction with other unmanned vehicles. 

The current trend of employing unmanned vehicles in the battlefield has extended 
beyond a single vehicle operation and is moving towards the gathering of real-time 
information using multiple unmanned vehicles. This could potentially impact a wide 
variety of military missions. The employment of multiple types of unmanned vehicles is 
a challenge for developers for a number of reasons. Eirst, there is the communication 
necessary between the vehicles. Since the vehicles are generally small, the 
communication equipment carried on the unmanned vehicles is usually size- and power- 
limited therefore resulting in corresponding limitations in bandwidth and range. Second, 
new guidance controls are necessary for developing collaborative behaviors between 
unmanned platforms. Third, there are always logistic and maintenance issues associated 
with using an unmanned vehicle and this is multiplied by the number of vehicles used. 

The control of a single unmanned vehicle has been explored intensively and much 
success has been achieved. These successes have been demonstrated in the obstacle 
avoidance and re-routing capabilities of unmanned vehicles like Micromouse [1], a 
competition that started in the late 1970s, with small robot mice solving a 16 x 16 maze 
autonomously. Seydou SOEIMARE in [2] has discussed, in depth, the uses of active 
vision sensors and laser sensors for real-time obstacle detection and avoidance in an 
indoor environment. Juan Carlos Rosete Eonseca in [3] has discussed the use of obstacle 
avoidance in the path planning of polar robots that allows the unmanned polar robots to 
perform path planning for greater autonomy and robustness. These discussions were 
based on indoor or laboratory environment studies that are not suitable for employment in 
the harsh environment of military operations. 
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Matthew Spenko in [4] proposed the eoncept of using trajectory space for high 
speed hazard avoidance in rough terrain. The concept proposed uses a compact 
framework for analyzing an unmanned ground vehicle’s (UGV’s) dynamic performance 
on uneven, natural terrain. The trajectory space defines the UGV’s performance limits 
based on vehicle parameters, terrain features, and hazard (obstacle) properties that are 
applicable to military operations. However, these are limited to single vehicle operations. 

The obstacle avoidance and path planning technology has extended beyond the 
UGV to UAVs as presented by Stephen Griffiths in [5]. These articles presented the 
ability to incorporate obstacle and terrain avoidance into real-time path planning by 
miniature aerial vehicles (MAVs), while taking into consideration the limitation of the 
MAV, which requires moving at a speed of 10 - 20 m/s to maintain flight. The limited 
payload of a MAV, as addressed in the articles, restricts the size of the onboard computer 
and the sensors that are employed to effectively survey the environment. 

Much work has also been done in the area of obstacle avoidance and path 
planning by an underwater vehicle as discussed by Yvan Petillot in [6]. These articles 
proposed a framework for the segmentation of sonar images, tracking of underwater 
objects, and motion estimations for obstacle avoidance and path planning by an 
underwater vehicle. Douglas P. Horner in [7] has also successfully demonstrated the 
ability to conduct underwater surveys and the avoidance of uncharted obstacles using 
Autonomous Underwater Vehicles (AUVs). The above discussions have been centered on 
individual platforms and do not address the problem of integrating the various platforms 
for an integrated mission. However, with the maturity of autonomous technology on each 
individual platform, there is an impetus for further research in the area of collaborative 
operations of various platforms to perform a specific mission. 

This thesis looks into the problems of developing a new guidance control law for 
a UAV to operate collaboratively with an AGV. The purpose of integrating a UAV with 
an AGV is to enhance the effectiveness of the AGV by providing an extended field of 
view through the UAV. The UAV, with its high maneuverability and altitude, is able to 
provide the advantage of forward reconnaissance for the ground vehicles. Another 
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advantage of using the UAV is the ability to provide a better look-ahead stand-off 
position and thus enable the ground vehiele to minimize contact with potential ground 
threats. 

In this scenario, the AGV is used as a forward ground reconnaissance vehicle. Its 
purpose is to identify potential threats to the trailing convoy. The ability to integrate the 
UAV and the AGV effectively will mean that the AGV would be able to tap into the 
advantage of stand-off reconnaissance by the UAV, thus allowing the AGV to have a fast 
response to any obstacles and a longer time to effectively change its planned mission or 
path. The stand-off reconnaissance ability would provide advance warning to ground 
vehicles on ambushes within its path, and thus increase the survivability of the ground 
vehicle and allow better force preservation. 

For the vehicles to work collaboratively, the UAV guidance law must compensate 
for the maneuvering AGV. Vladimir N. Dobrokhodov, in [8], has discussed extensively 
the influence of the AGV ground speed on the performance of the UAV following the 
flight path generated. The proposed concept of a circular flight path for the UAV could 
be adapted for the integration of a UAV and an AGV, which would allow for speed 
compatibility/adjustment of the fast-flying UAV with the slower-moving AGV. This 
concept will be used in the construction of the simulation in this thesis. 

Another problem associated with integration of the UAV and the AGV is the 
ability to synchronize its routes and path planning algorithm to allow better autonomy of 
the vehicles, such that the UAV and the AGV could operate independently when 
communication between the vehicles is disrupted. This would allow the UAV to predict 
the location of the AGV based on the mission profile. The synchronization of routes and 
path planning algorithm will not be discussed in this thesis. 

The development of computer vision algorithm for object detection and tracking 
enables the autonomous vehicles to detect obstacles and to track a target. This would 
greatly improve the autonomy of the autonomous vehicles and allows for minimum 
operator interference. The computer vision for autonomous vehicles will not be 
addressed in this thesis. 
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Communication and collaboration between the UAV and the AGV is of 
paramount importance to the suecess of a forward ground reeonnaissance for a trailing 
convoy mission. The limited payload allowed on the UAV poses as a problem to 
ineorporate a larger and higher power transmitter for long range communieation. Thus the 
communieation range between the vehicles is limited by the capability of the 
communieation equipment that is carried by the UAV. This was highlighted by Stephen 
Griffiths in [5]. One solution that is proposed by Douglas P. Homer [9] is the use of an 
artifieial potential field for UAV guidance and optimization of WLAN eommunieation, 
whieh would optimize the communieation between the UAV and a series of AGVs using 
the signal strength ratio. Another method discussed in this report is in using the reeeived 
signal strength to form a linked control to limit the extended range of the UAV; such 
control mechanisms are referred to as “tether control” in this thesis. 

The tether control uses the reeeived signal strength between the UAV and the 
AGV to effeetively control the flight path of the UAV and ensure reliable communieation 
between the two vehicles. The method employed essentially makes use of the received 
signal strength to form an artificial dome around the AGV to demareate the limits of 
reliable communication. If the UAV flies beyond the dome, communieation is assumed to 
be dismpted, thus the artificial dome contains the furthest flight path of the UAV to be 
within this field. 

The effectiveness of the UAV greatly depends on its ability to keep the onboard 
camera pointing at a fixed distance forward of the AGV. The camera system available on 
the UAV in the study has a zoom eontrol and a two-degree-of-freedom (DOF) motion, 
namely, pan and tilt. The UAV in flight has a six-DOF and it is essential that the camera 
is able to operate in such a manner that it complements the state condition of the UAV to 
effectively point the eamera at a fixed position forward of the AGV. Vladimir N. 
Dobrokhodov in [8] has discussed the use of vision-based tracking and motion estimation 
for moving targets using small UAVs. The eoneept proposed integrates UAV gimbal 
control with guidance of the UAV that allows tracking of a moving coordinate ground 
vehieles. This eoneept requires several geometrical transformation mathematical 
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computation, which is within the capability of the computer onboard the UAV. This 
thesis will adapt the algorithm discussed for the implementation flight control of the 
UAV. 

The pin-hole camera model was used for this thesis, where the image captured is 
projected onto the earth’s surface. A working assumption for the thesis is that the surface 
of the earth is considered to be a plane and three-dimensional earth surface model is 
recommended for future work. At high elevation and tilt angles, a larger area will be 
captured at the far-field and a smaller area at the near-field. Such effect is commonly 
known as the keystone effect of the camera, which is discussed by Andrew Woods in 
[10]. The ability to accurately demark the keystone effect will allow the image 
processing unit to interpret the captured image as illustrated in Figure 1. The model 
provides a basis for the image processing unit to correct the error and thus improve the 
accuracy of the data. 
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This thesis was set forth to research the problem associated with integrating the 
existing UAV (Scan Eagle) with an AGV to provide real-time communication between 
the vehicles in a truly cooperative environment in order for the vehicles to achieve their 
mission. The model uses Matlab Simulink Version 6.4 (R2006a) [11] to simulate the 
UAV flying in six-DOF and incorporate the onboard camera to point at a fixed distance 
forward of the AGV. The AGV in the simulation has two-DOF and is programmed to 
travel on a road using a GeoTiff Map [12 and 13]. 

This thesis uses the operational concept of an AGV traveling along a road and a 
UAV is tethered to the AGV to provide reconnaissance images of a fixed distance 
forward of the AGV as shown in Figure 2. 

FOVofUAV 



Latitude Longtitude 

Figure 2. Operational Concept 

The vehicles are equipped with standard WiFi 802.11 communication equipment 

(ITT Mesh Card) that has a limited range of communication. The assumption made in for 
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this thesis is that the signal-to-noise ratio (SRN) of the communication equipment is a 
linear function of distance. The figure above shows the field of view (FOV) of both 
vehicles. The AGV has a limited FOV due to the obstruction from the natural feature on 
the ground, however, the UAV, due to its operating altitude, is able to acquire a better 
FOV and identify a possible ambush site. The UAV images are transmitted to the AGV 
for further analysis. As a result of the analysis, the AGV can change its planned path or 
prepare for a counter attack. 

The UAV used for discussion in this thesis is the newly acquired NFS Scan Eagle 
developed by Boeing. It weighs 18 kg and is designed for a continuous mission of more 
than 15 hours, with a cruising speed of 50 knots, and at maximum altitude of 5,000 
meters. Scan Eagle has a carrying capacity of 6 kg and can be launched and retrieved 
over any terrain including onboard a ship during naval operations. The UAV system 
includes a Sky Wedge hydraulic launcher. Sky Hook retrieving system, and a mobile 
ground control element as shown below. 



Figure 3. NFS Scan Eagle 


7 







Figure 4. Scan Eagle Launching System 



Figure 5. Scan Eagle Recovery System 


8 










Scan Eagle was designed with a removable avionics bay and two expansion slots 
allowing seamless payload integration. Its vision system incorporates a Sony FCB- 
EX780S camera with two-axis stabilization. The eamera line-of-sight is gyro-stabilized 
with a 5-hertz bandwidth, and further eleetronieally stabilized by Sony’s SteadyShot^'’^ 
with a 3 to 20 hertz bandwidth. The camera has a eontinuous pan capability utilizing 
slip-rings and a tilt angle of positive 30 degrees to negative 110 degrees. The eamera has 
a 25:1 optical zoom from 45 degrees to 1.8 degrees FOV and a 12:1 digital zoom from 
1.8 degrees to 0.15 degrees. The eamera is able to operate in stabilized line-of-sight 
(EOS), waypoint tracking, or target tracking modes. 
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II. THEORETICAL APPROACH TO MODELING THE TETHER 
OPERATION OF THE UAV AND AGV 


A, OVERVIEW OF THE MODEL 

The approached taken for this thesis is based on modeling using Simulink Version 
6.4 (R2006a) as shown in Figure 6. The model is divided into five components, namely, 
the six-DOF model for the UAV, the camera gimbal control model, the camera keystone 
model, the UAV flight path control model, and the tether control model. The model is 
adapted from the work done in [8]. 



Figure 6. Overview of Model [After: 8] 

B. UAV SIX-DOF FLIGHT MODEL 

The UAV six-DOF model served as a basis for the simulation as it provides the 
required state condition of the UAV for the model to operate. Brian L. Stevens in [14] 
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discussed in detail the methodology for modeling the state eondition for an aireraft, 
which is applicable to the modeling of the UAV. Figure 7 shows the UAV six-DOF 
model for the simulation. The model is eomprised of three components, namely, the 
Autopilot, Forees and Moments, and the Six-DOF equations of motion model. 



Figure 7. UAV Six-DOF Model 


The autopilot model generates the required elevator, aileron, rudder deflection and 
throttle control using the state eondition (roll, pitch, yaw, position, and velocity) of the 
UAV based on the yaw, piteh command input. The autopilot eonsists of several control 
loops providing stabilization of attitude, altitude and the speed of the UAV. 

The Forees and Moments model receives inputs from the outputs of the autopilot 
and couples it with the state condition of the UAV to generate the forees and moments 
acting on the UAV. The forees and moments model essentially takes the elevator, aileron 
and rudder defleetion, and based on the state condition, resolves the forees into three 
eomponents, namely, the X (Fx), Y (Fy) and Z (Fz) components, with respect to the body 
frame eoordinates of the UAV as shown in Figure 8. It ealeulates the moments generated 
by these forees on the three prineipal body axes of the UAV, as shown. 
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Figures. Forces and Moment on a UAV 


The moments (Tb) generated by the inertial and combination of forces and motion 
of the UAV in the airframe-body coordinate (ABC) are calculated based on the following 
equations: 

Tb = Hb + ®bxHb (1) 

where Hb is the angular momentum vector, oob is the absolute angular velocity vector in 
ABC as follows: 



“P" 


(j)- 

\j/sin0 


®B = 

Q 

= 

0 cos (() + \j/cos Osin (j) 


R 


\j/cos0cos(t)-0sin(t) 


"PJ 

X 

-QJxy 

-RJxz" 


Hb = 

QJy 

-RJyz 

-PJxy 



_RJz 

-PJxz 

-QJyz 



( 2 ) 


( 3 ) 


The angular velocity vector is a composite of P (roll rate), Q (pitch rate) and R 
(yaw rate) of the UAV. The (j) (roll), 0 (pitch), \|/ (yaw), are the Euler angles and ^, 0 
and \j/ are the rate of change of the Euler angles. J is the 3x3 tensor of inertia of the 
UAV, comprising 9 individual components. 
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The Six-DOF equations of motion model uses the forees and moments inputs, 
coupled with the current state condition of the UAV to compute the next state condition. 
The simulation uses a flat earth model (North-East-Down, NED), which is sufficiently 
accurate for the current application. The flat earth model ignores the earth’s rotation (©e) 
terms and uses the Navigation Equation (4), Eorce Equation (5) and Moment Equation (6) 
to compute the states: 


p = bX 

(4) 

Vb =irFB-®BXVB 

(5) 

Cbg = -J^' (cOg X (JCOg )) -t J^^Tg 

(6) 


The navigation equation provides the derivative of the position in the NED 
coordinate, where B is the transformation matrix for ABC to the NED coordinate and Vb 
is the absolute velocity of the UAV in ABC. The forces equation gives the derivative of 
velocity in ABC, where Eb is the applied forces in ABC. The moment equation gives the 
derivative of the absolute angular velocity vector in ABC, where J is the inertia matrix as 
follows: 


J = 


Jx 

Jy 

Jz 




(V) 


Integrating the derivative of the position, velocity, and absolute angular velocity 
vector will provide the absolute position in the NED coordinate, velocity and absolute 
angular velocity vector in ABC of the UAV. These would be used for the simulation. 


C. UAV FLIGHT PATH CONTROL ALGORITHM 

The UAV in flight is assumed to travel at a higher speed compared to the AGV 
and it is crucial to control the flight path in accordance with the average speed of the 
AGV. The flight path control algorithm achieved this by commanding the UAV to fly in 
a circular motion with respect to an imaginary moving target called the UAV CG. The 
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UAV CG is controlled by the tether eontrol algorithm to achieve a constant 
communication link between the UAV and the AGV. The flight path of the UAV is 
illustrated in Figure 9 below with a moving UAV CG. 


XYPW 



Figure 9. Flight Path of UAV with Moving UAV CG 


The UAV flight path eontrol algorithm adapted the eontrol algorithm proposed in 
[7] and is shown in Figure 10. 
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From Figure 10, Vg is the UAV ground speed, ?Lg is the line of sight (LOS) 
veetor, Xp is the veetor perpendieular to Xg, s is the angle between the LOS veetor and 
the camera heading, X is the LOS angle, \|/ is the UAV heading, \\f^^ is the gimbal pan 
angle , r\ is the angle between the Vg and Xp vectors, V, is the speed of the imaginary 
target and \|/, is the heading of the imaginary target. The tracking problem kinematics 
for the target are given by; 


V COST]- V, cos(\|/,-(\|/-ri)) . 

T] =- 2 -^-^ + \i/ 

P 

(8) 

. V COST! -Vj COS (\|/j -(\|/-ri)) . . 

8= V Vh 

p 

(9) 

p = - Vg sin ri + V, cos (\|/, - (\|/- p)) 

(10) 
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p is the range from the UAV to the imaginary target. The objective of the control 
system is to drive s and r) to zero using the UAV turn rate \j/ and pan rate ij/^ as control 
inputs, thus the control law is as follows: 

V 

vj; = — cosTi-kjTi (11) 

Ph 

\j/h =kjri + k 28 (12) 

where pj^ is the desired horizontal range to the target as user-adjustable input variables 
and, ki and ki are gains for the control. 

The UAV flight path control model results in the UAV flying in a circular path 
with respect to an imaginary reference point on the ground (UAV CG). The camera 
vector shown in Figure 10 is an imaginary camera and does not represent the actual 
onboard camera of the UAV. However it is used for the control law development. The 
coordinated (camera and UAV guidance) control law eliminates the need for the AGV to 
constantly control the flight path of the UAV and reduce the computation burden of the 
AGV. It also allows the UAV greater flexibility to modify its flight path to avoid 
obstacles or adjust its height to avoid detection. The control law also allows greater 
autonomy to the UAV and thus forms the backbone for the successful implementation of 
the tether control. The onboard camera is controlled by a separate control algorithm 
called the camera gimbal control algorithm. 


D. CAMERA GIMBAL CONTROL ALGORITHM 

The camera gimbal control algorithm allows the camera to operate independently 
from the flight path of the UAV. The camera is programmed to point on the road a fixed 
distance in front of the AGV. This is achieved by giving a position coordinate called the 
“target” to the camera gimbal control model as illustrated in Figure 11. The camera is 
initially set to zero degree pan and tilt. When a target coordinate is received, the camera 
gimbal control model will compute the required pan and tilt angle for the camera gimbal 
servo mechanism to position the camera. Refer to Appendix A for the camera gimbal 
control model. 
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From Figure 11, , A^, and A^ were defined as the relative position of the UAV 

and the ground target in ABC, (t)p and (j), is the desired pan and tilt angle of the camera. 
The objective of the control law is to provide the pan and tilt error, Spand 8,, to a 

proportional and integral controller, which in turn, commands the camera to adjust its 
position as follows: 


i = tan 
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V Vabc 


(()j = tan ' 
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^ / ABC 
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(13) 
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The control algorithm uses the relative position of the UAV and ground target in 
the NED coordinate and translate them to ABC using the following formula; 
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(17) 


Xt, Yt and Zt is the position of the ground target, Xu, Yu and Zu is the position of 
the UAV in the NED coordinate and B is the Direct Cosine Matrix for transformation 
from the NED coordinate to ABC as follows: 


B = 


COS0COS\)/ 

sm(j)sin0cos\j/-cos(t)sin\j/ 
cos (j) sin 0 cos \|/ + sin (j) sin \|/ 


cos 0 sin \)/ 

sin (j) sin 0 sin \j/ + cos (j) cos \|/ 
cos (j)sin 0 cos\|/- sin (j) cos\|/ 


-sin0 
sin (j) cos 0 
cos (j) cos 0 


(18) 


This control allows the camera to track an independent ground target, taking into 
consideration the state position of the UAV, thus decoupling the flight control of the 
UAV from the camera. 


E, CAMERA KEYSTONE MODEL 

Due to the position and focusing of the camera on the target at an elevated angle, 
the captured image will be distorted because of the keystone effect. Understanding the 
keystone effect will allow the image processing of the captured picture to be more 
accurate. The model that simulates the keystone effect is illustrated in Eigure 12. Refer 
to Appendix B for the camera keystone model. 
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Figure 12. Modeling the Camera Keystone Effect 

0n is the unit vector of the camera. Using the zoom angle of the camera (0z), four 
unit vectors (0a, 0b, 0c and 0d ) could be derived to point at the four comers. A, B, C 
and D as shown in Figure 12. The point interceptions of the four unit vectors with the 
surface will give the coordinates of A, B, C and D. The dark shaded region is the area 
captured by the camera image. 

The assumption of a pin-hole camera is used for the camera keystone model 
where the image captured is assumed to be proportional to the image that the camera sees 
as illustrated in Figure 13 below. The light from the actual object passes through the pin¬ 
hole and is projected on the back of the box as the image is captured. The distortion on 
the captured image, due to the lens of the camera, is not taken into consideration for the 
simulation. 
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Object Image 



Actual Object 

Figure 13. Pin-hole Camera Assumption 

F. TETHER CONTROL FOR EFFECTIVE COMMUNICATION 

The tether control model creates an invisible tether link between the UAV and the 
AGV to ensure that the UAV is flying at a predictable distance away from the AGV. 
This distance is determined by the received signal strength of the communication 
equipment on the UAV and the AGV. The communication equipment used on both 
vehicles for the exchange of information is WiFi 802.11, the wireless communication 
bandwidth. It is assumed that each vehicle has the capability to monitor the strength of 
the received signal, and using the received signal strength from the AGV, the UAV is 
able to adjust the distance to the AGV to ensure and maintain good communication. The 
SNR measurement of the radio signal from the UAV is loosely defined as an “invisible 
tether” since it is the SNR that determines the maximum distance the UAV can fly from 
the AGV. 

The tether control model uses the received signal strength and the UAV LOS to 
the AGV to determine the distance of the invisible tether and the desired horizontal range, 

Ph- 

Ph=kZu (19) 

The k value depends on the flight characteristics of the UAV and is selected via 
trial and error. The desired horizontal range increases proportionally with the altitude of 
the UAV to ensure the stability of the UAV flight path control law. The tether control 

concept is illustrated in Figure 14. It should be noted that the received signal strength 
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used in this model is an operator input and is the mean of the received signal strength. 
The actual received signal strength might fluctuate constantly due to the presence of noise 
in the signal, and special care has to be taken to filter the unwanted noise. Refer to 
Appendix C for the simulation model. 



From Figure 14, Rs is the maximum communication range to ensure reliable 
communication, Sn is the received signal strength from the onboard communication 
equipment, and Rh is the horizontal vector of Rs. For this thesis, an operator input value 
for Sn is used for the simulation and Rs is determined by the equation below: 

Rs=k,S^ (20) 

kc is the proportional gain which should be set by the operator for optimum value 
of Rs, which has the longest possible distance with reliable communication. The tether 
control uses the altitude of the UAV to determine the optimum pj^ using a gain controller 
as follows: 
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( 21 ) 


P = 

^CG 


Ys 


where Pcg is the UAV CG position and Xg, Yg and Zg is the position of the AGV in the 
NED. For the tether eontrol to be effeetive, the seleetion of the k value is important to 
ensure that the UAV flight path is stable 


G. INTEGRATING THE MODEL 

The UAV six-DOF flight model aets as the baekbone for the simulation. It 
provides the state eonditions of the UAV that are used by the rest of the model. The 
AGV and the desired fixed distanee forward of the AGV are designed to travel on a road 
represented by a series of disereet eoordinates along the route with one meter spaeing. 
These eoordinates are based on a relative position refereneed to a fixed point on the 
GeoTiff maps. The speed and starting loeation of the AGV and the forward fixed 
position are eontrolled by the user in the model. 

The tether eontrol model uses the position of the UAV and the AGV, the reeeived 
signal strength, and the UAV altitude to eompute both the UAV CG loeation and the 
desired horizontal range for the UAV. The UAV CG loeation and the desired horizontal 
range are passed into the UAV flight path eontrol model to generate the desired flight 
path for the UAV. The UAV flight path model outputs the piteh and yaw eommand for 
the UAV six-DOF model to eompute the state eondition of the UAV. 

The flight path of the UAV, with respeet to a moving UAV CG loeation, is shown 
previously in Figure 9. It shows that the UAV is traveling in a eireular path to 
eompensate for the slower speed AGV. An alternative, eommanding the UAV to adopt an 
“S” eurve flight path to eompensate for the slower speed AGV, has been explored and is 
illustrated in Figure 15. Sueh a flight path was deemed ineffeetive and undesirable as the 
error generated for the eamera during the turn is large and eaused the eamera to 
momentarily lose its foeus on the desired loeation. 
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Figure 15. “S” Curve Flight Path 

The camera gimbal controller uses the state condition of the UAV from the UAV 
six-DOF model and the hxed position forward of the AGV to determine the pan and tilt 
angle for the camera. The pan and tilt angle is sent to the camera keystone model to 
derive the positions that the field of view of the camera projects onto the ground. 

The model uses the GeoTiff map of the Camp Roberts, CA area (in the vicinity of 
35.7N, 120.7E), for visualizing the simulation. The location of the UAV, the AGV, the 
fixed position forward of the AGV and the UAV CG are superimposed over the GeoTiff 
map to provide a sense of location. The keystone effect is shown on the GeoTiff map. 
The model uses relative position for ah computations and is subsequently translated into 
the GeoTiff map coordinates using a conversion factor for the location as follows: 
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Latitude conversion factor = 111119.99966 m/degree 

Longitude conversion factor = 91023.79479 m/degree 

The conversion factors are applied to the x-coordinate and y-coordinate only. The 
z-coordinate is unaffected due to the flat earth assumption that the model is using. 
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III. SIMULATION AND RESULT 


The output of the simulation shows that the UAV is able to accurately point the 
camera at the fixed forward position of the AGV along the road while following the flight 
path generated by the flight path control algorithm. The outputs are shown in the 
following figures. 
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Figure 16. Output of Simulation 1 
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Figure 17. Output of Simulation 2 
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Figure 18. Output of Simulation 3 



28 

















35.75 

35.74 

35.73 

35.72 

35.71 

35.7 

35.69 

-120.81 -120.8 -120.79 -120.78 -120.77 -120.76 -120.75 
Figure 19. Output of Simulation 4 
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Figure 20. Output of Simulation 5 
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Figure 21. Output of Simulation 6 

From the figures above, it ean be seen that the UAV is flying in a cireular path 
around the UAV CG with a relative eonstant desired horizontal range. The AGV and the 
fixed position forward of the AGV are moving along the road shown in the GeoTiff Map. 
The area in which the image is captured by the onboard camera shows that the camera is 
able to track the fixed forward position effectively. The keystone effect is also shown. 

This simulation shows that the area captured by the camera image is constantly 
changing due to the position of the UAV. The area captured is bigger when the UAV is 
flying further away from the fixed forward position. This is undesirable and further work 
should be done to provide an active zoom control to create a constant area captured by 
the camera that is independent from the UAV position. 

The camera is out of focus from the fixed position forward of the AGV at the start 
of the simulation. This is due to the time lag of the camera pan and tilt mechanisms that 
are incorporated into the model. The initial pan and tilt angle of the camera is set at zero 
degrees, which requires the camera gimbal control algorithm to correct the camera pan 
and tilt angle in order to track the ground target. This is shown in Figure 24. 
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Figure 22. Camera Start-up Error 


The initial speed and heading of the UAV plays an important part for the UAV 
flight control algorithm. From Figure 23, it can be seen that the UAV is further from the 
UAV CG location. The relative position of the UAV and the UAV CG is larger than the 
desired horizontal range determined by the tether control algorithm. This is due to the 
start location of the UAV, which is set at 35.71N, 120.77E and the initial heading of the 
UAV is set to due east. It can also be seen that the UAV is correcting its path toward the 
UAV CG by changing its heading gradually to north. This gradual movement is due to 
the minimum turning radius that is related to the flight characteristic of the UAV. 
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Figure 23. UAV Initial Flight Correction 

From the simulation output above, it is noted that the location of the UAV CG 
does not follow a road. This is due to the fact that the UAV CG is an imaginary target 
location for the UAV flight control algorithm to take as a reference for planning the flight 
path of the UAV. The UAV CG location is determined by the tether control algorithm, 
given the signal strength of the communication between the UAV and the AGV. The 
location of the UAV CG is strategically positioned to be between the target location and 
the AGV to achieve the optimum effect of creating a flight path that is near the target and 
within communication range. 
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IV. CONCLUSION AND RECOMMENDATION 


The ability to integrate a heterogeneous mix of unmanned vehieles has great 
potential in enhaneing military operations and provides better real-time situation 
awareness of the battlefield. With the advaneements in unmanned vehieles ineluding 
high speed real-time obstaele avoidanee and intelligent path planning and eorreetion, 
there is a need to researeh the integration of various unmanned platforms to achieve a 
single mission. 

This thesis looks into the three problems associated with the integration of the 
UAV and the AGV: the development of an autopilot guidance strategy to compensate for 
the relative speed differences between the UAV and the AGV, the keystone effect of the 
image from projecting the camera’s field of view onto the ground given the pan and tilt 
from the onboard camera of the UAV, and the need to ensure the reliability of the 
communication network between the UAV and the AGV with the limited range of the 
communication equipment. 

The results show that these problems associated with the integration of the UAV 
and the AGV can be overcome. The first problem, the relative speed difference between 
the UAV and the AGV, was overcome by using a circular flight path for the UAV to 
compensate for the slower speed AGV. The second problem, associated with the 
distortion of the captured image due to the keystone effect of the camera, was addressed 
by a model to identify the actual area captured by the image. The third problem, which 
arose from the need to provide reliable communication between the UAV and the AGV 
with the limited communication range of the equipment, was overcome by implementing 
a tether control between the UAV and the AGV. 

From the result of this thesis, where the UAV is able to integrate with the AGV, 
this concept could be used for the UAV to integrate with an Unmanned Surface Vehicle 
(USV) to provide forward reconnaissance for the USV when traveling along a river. 

From this research, it is recommended that a target search model be incorporated 
to allow the UAV to conduct a search along a series of routes that lead to the 
advancement of the AGV. This will provide preemptive warnings to the AGV on 
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possible obstacles to be encountered, for example, a vehicle traveling toward the planned 
route of the AGV that might cause the obstruction of the AGV. The search model could 
take reference from the Rapidly-exploring Random Trees (RRT)-Connect proposed by 
James J. Kuffner in [15]. The RRT is a simple and efficient randomized algorithm for 
solving single-query path planning problems in high-dimensional configuration spaces. 
The RRT-Connect provides an effective way to conduct path planning and identify an 
alternative route, where such a route could pose as an avenue for other vehicles that could 
travel into the planned path of the AGV and disrupt the operation. 

A dynamic road demarcating algorithm is recommended to replace the sets of 
discreet coordinates representing the road in the current model. The simulation used a set 
of discreet predetermined coordinates to identify the road network on the GeoTiff Map. 
The nature of military operations frequently entails the creation and destruction of roads, 
thus incorporating a dynamic road demarcating algorithm would be attractive to any 
military operation. 

It is also recommended that an active zoom control for the camera be incorporated 
into the camera control model to enhance the stability of the captured video by 
maintaining a relatively constant area of view from the camera. 

While this thesis looked into using the UAV to provide forward reconnaissance 
for the AGV, the roles can be reversed. The AGV or USV could provide close-in 
surveillance on positions of interest that are determined by the UAV. Such an application 
is useful in a scenario where the UAV has identified a position or object of interest and 
requires further investigation or physical presence on the position or object. In locations 
where the position of the tracked target is occluded, the AGV can maintain position lock 
until the UAV can continue its tracking. 

Finally, the UAV can be used for effective, real-time path planning for the AGV 
in built-up areas. The UAV, with its unobstructed flight path at high altitude, could 
provide real-time information for the AGV path planning. This would effectively reduce 
the possibility of detection of the AGV by the enemy, and thus improve survivability. 
One problem that is associated with UAV operation in built-up areas, is the high 
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probability of losing the autonomous tracking of the target, thus further research is 
necessary to explore the effects of such occlusion and methods need to be devised to 
overcome this problem. 

In conclusion, the work done in this thesis has shown the possibility and the 
potential of integrating heterogeneous unmanned vehicles. The collaborative nature of 
integrating the unmanned vehicles would inevitably enhance the effectiveness of the 
vehicles in a wide variety of applications. 
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APPENDIX A. CAMERA GIMBAL CONTROL MODEL 



A, CAMERA PAN/TILT MODEL 


Relative Position 


atan2(u[2].u[1]) 
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APPENDIX B. CAMERA KEYSTONE MODEL 


A. CAMERA MODEL 
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B, 


CAMERA ANGLE ABCl 



C “TGTTRACKING.M” LISTING 

function [sys,xO,str,ts] = sldemo_tanksfun(t,x,u,flag) 

switch flag, 

% Initialization % 
case 0, 

[sys,xO,str,ts]=mdllnitializeSizes; 

% Update % 
case 2, 

sys=mdlUpdate(t,x,u); 

% Unhandled flags % 
case {1, 3, 4, 9 } 
sys=[]; 

end 

% end sidemo tanksfun 


% Return the sizes, initial conditions, and sample times for the S-function. 

function [sys,xO,str,ts]=mdllnitializeSizes 

sizes = simsizes; 

sizes.NumContStates = 0; 

sizes.NumDiscStates = 0; 

sizes.NumOutputs = 0; 

sizes.Numinputs = 18; 

sizes.DirFeedthrough = 1; 

sizes.NumSampleTimes = 1; 

sys = simsizes(sizes); 


% initialize the initial conditions 
xO = []; 

str = []; % str is always an empty matrix 


% initialize the display figure 
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ts = [0 0]; 
figure (1) 

mapshow('351 20f72.tif); 
hold on; 

xlim=[-120.79 -120.75]; 
ylim=[35.70 35.73]; 

% end mdllnitializeSizes 


0 /„ = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 

% Handle discrete state updates, sample time hits, and major time step requirements. 
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 

function sys=mdlUpdate(t,x,u) %#ok 

% initialize the return arg 
sys=[]; 

TgtX=u(1); 

TgtY=u(2); 

TgtZ=u(3); 

CamX=u(4); 

CamY=u(5); 

CamZ=u(6); 

CGX=u(7); 

CGY=u(8); 

CGZ=u(9); 

UAVX=u(10); 

UAVY=u(11); 

UAVZ=u(12); 

CamZm=u(13); 

Pan=u(14); 

Tilt=u(15); 

GVX=u(16); 

GVY=u(17); 

GVZ=u(18); 

D=sqrt((CamX-UAVX)''2+(CamY-UAVY)''2+(CamZ-UAVZ)''2); 

D1=D*sin(Tilt)/sin(pi-CamZm-Tilt); 

D2=D*sin(pi-Tilt)/sin(Tilt-CamZm); 

U1=D*sin(CamZm)/sin(pi-CamZm-Tilt); 

U2=D*sin(CamZm)/sin(Tilt-CamZm); 

R=D*tan(CamZm); 

R1=D1*tan(CamZm); 

R2=D2*tan(CamZm); 

Pan=atan2((CamY-UAVY),(CamX-UAVX)); 

Tilt = atan2(sqrt(((CamY-UAVY)''2)+((CamX-UAVX)''2)),(CamZ-UAVZ)); 

Lat0=35.7114; %Y Datum 

Lon0=-120.77256; %X Datum 

LatCov=111119.99965975754; %m/deg for Latitude 

LonCov=91023.7947885214; %m/deg for Longtitude 

Focx(1)=((CamX-U1*cos(Pan))+(R1*sin(Pan))); 

Focx(2)=((CamX+U2*cos(Pan))+(R2*sin(Pan))); 

Focx(3)=((CamX+U2*cos(Pan))-(R2*sin(Pan))); 

Focx(4)=((CamX-U1*cos(Pan))-(R1*sin(Pan))); 
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Focx(5)=((CamX-U1*cos(Pan))+(R1*sin(Pan))); 


Focy(1)=((CamY-U1*sin(Pan))-(R1*cos(Pan))); 

Focy(2)=((CamY+U2*sin(Pan))-(R2*cos(Pan))); 

Focy(3)=((CamY+U2*sin(Pan))+(R2*cos(Pan))); 

Focy(4)=((CamY-U1*sin(Pan))+(R1*cos(Pan))); 

Focy(5)=((CamY-U1*sin(Pan))-(R1*cos(Pan))); 

FocLon=LonO+Focx/LonCov; 

FocLat=LatO+Focy/LatCov; 

CGLon=LonO+CGX/LonCov; 

CGLat=LatO+CGY/LatCov; 

UAVLon=LonO+UAVX/LonCov; 

UAVLat=LatO+UAVY/LatCov; 

TgtLon=LonO+TgtX/LonCov; 

Tg tLat= LatO+TgtY/LatCov; 

GVLon=LonO+GVX/LonCov; 

GVLat=LatO+GVY/LatCov; 

hold on 

h=findobj('Color','r'); 

delete(h); 

h=findobj('Color','b'); 

delete(h); 

h=findobj('Color','m'); 

delete(h); 

h=findobj{'Color',y); 

delete(h); 

h=findobj('Color';k'); 

delete(h); 

h=findobj('Color';c'); 

delete(h); 

plot(TgtLon,TgtLat,'+r'); 
plot(FocLon,FocLat,'b'); 
plot(CGLon, CGLat.'ok', 'MarkerFaceColor'.'y'); 
plot(UAVLon, UAVLat,'om','MarkerFaceColor','g'); 
plot(GVLon, GVLat,'sr','MarkerFaceColorVc') 

h = legend('Ground Target'/Camera Focus Point'/UAV CG'.'UAV','Ground Vehicles', 3); 
legend('Ground Target','Camera Focus Point','UAV CG','UAV','Ground Vehicles',4); 
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APPENDIX C. TETHER CONTROL MODEL 


A. TETHER CONTROL MODEL 
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B. GROUND VEHICLES MODEL 
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C. TARGET MODEL 





45 



























THIS PAGE INTENTIONALLY LEET BLANK 


46 



LIST OF REFERENCES 


1. Micromouse Competition Website: http://mieromouse.eannoek.ae.uk/index.htm . 
Deeember 2006. 

2. Seydou SOUMARE, Akihisa OHYA and Shin’iehi YUTA, “Real-Time Obstaele 
Avoidanee by an Autonomous Mobile Robot using an Aetive Vision Sensor and a 
Vertieally Emitted Laser Slit,” The 7th International Conferenee on Intelligent 
Autonomous Systems (IAS-7), California, Mareh 25-27, 2002. 

3. Juan Carlos Rosete Fonseea, Efren Gorrostieta Hurtado, Joaquin Perez Meneses, 
“Obstaeles Avoidanee in a Self Path Planning of a Polar Robot,” CERMA, pp. 
157-162, Eleetronies, Robotics and Automotive Meehanies Conferenee (CERMA 
■06), 2006. 

4. Matthew Spenko, Karl lagnemma, Steven Dubowsky, “High Speed Hazard 
Avoidanee for Mobile Robots in Rough Terrain,” Published in Gerhart, Grant R.; 
Shoemaker, Chuek M.; Gage, Douglas W., eds. Unmanned Ground Vehiele 
Teehnology VI. Proeeedings of the SPIE, Volume 5422, pp. 439-450 (2004). 

5. Stephen Griffiths, Jeff Saunders, Andrew Curtis, Tim MeLain and Randy Beard 
“Obstaele and Terrain Avoidanee for Miniature Aerial Vehieles,” In Review, 
IEEE Roboties and Automation Magazine (2006). 

6. Yvan Petillot, loseba Tena Ruiz, and David M. Lane, “Underwater Vehiele 
Obstaele Avoidanee and Path Planning Using a Multi-Beam Forward Looking 
Sonar,” IEEE J. Oeeanie Eng., vol. 26, pp. 240-251, April 2001. 

7. Douglas P. Horner, Anthony J. Healey, Sean. P. Kragelund, “AUV Experiments 
in Obstaele Avoidanee,” Proeeedings of IEEE Oeeans 2005 Conferenee. 

8. Vladimir N. Dobrokhodov, Isaae I. Kaminer, Kevin D. Jones and Reza 
Ghabeheloo, “Vision-Based Traeking and Motion Estimation for Moving targets 
using Small UAVs,” Proeeedings of Ameriean Control Conferenee, Minneapolis, 
June 2006. 

9. Douglas P. Horner, Anthony J. Healey, “Use of Artffieial Potential Fields for 
UAV Guidanee and Optimization of WLAN Communieations,” Proeeedings of 
the IEEE AUV2004 Conferenee, Maine, June 2004. 

10. Andrew Woods, Tom Doeherty and Rolf Koeh, “Stereoseopie Displays and 
Applieations IV,” Proceedings of the SPIE Volume 1915, San Jose, CA, Feb. 

1993. 

11. Eva Part-Enander, Anders Sjoberg, Bo Melin, and Pemilla Isaksson, “The 
MATLAB Handbook,”Addison-Wesley, 1998. 


47 



12. Sk. Sazid Mahammad and R. Ramakrishnan, “GeoTiff - A Standard Image File 
Format for GIS Applications,” Space Application Centre, ISRO, Ahemedabad. 

13. Dr. Niles Ritter and Mike Ruth, “GeoTiff Format Specification - GeoTiff 
Revision 1.0,” SPOT Image Corp., (2000). 

14. Brian L. Stevens and Frank L. Lewis, “Aircraft Control and Simulation,” 2nd 
edition, Wiley, pp. 59-142, 2003. 

15. James J. Kuffner, Jr. and Steven M. LaValle, “RRT-Connect: An Efficient 
Approach to Single-Query Path Planning,” Proceeding of 2000 IEEE International 
Conference on Robotics and Automation, vol. 2, pp. 995-1001, 2000. 


48 



INITIAL DISTRIBUTION LIST 


1. Defense Teehnieal Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate Sehool 
Monterey, California 

3. Prof. Anthony J. Healey 

Department of Meehanieal and Astronautieal Engineering 

Naval Postgraduate Sehool 
Monterey, California 

4. Prof. Douglas Horner 

Department of Meehanieal and Astronautieal Engineering 

Naval Postgraduate Sehool 
Monterey, California 

5. Prof Vladimir Dobrokhodov 

Department of Meehanieal and Astronautieal Engineering 

Naval Postgraduate Sehool 
Monterey, California 

6. Prof Yeo Tat Soon 

Direetor of Temasek Defenee Systems Institute (TDSI) 
Viee-Dean, Eaeulty of Engineering, Admin 
National University of Singapore 
Singapore 

7. TanEaiPoh 

Senior Admin Offieer (MDTS) 

Temasek Defenee Systems Institute (TDSI) 

National University of Singapore 
Singapore 


49 



